Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training

Understanding mathematical questions effectively is a crucial task, which can benefit many applications, such as difficulty estimation. Researchers have drawn much attention to designing pre-training models for question representations due the scarcity of human annotations (e.g., labeling difficulty). However, unlike general free-format texts user comments), are generally designed with explicit purposes and logic, usually consist more complex content, formulas, related knowledge Function). Therefore, problem holistically representing remains underexplored. To this end, in paper, we propose novel contrastive approach representations, namely QuesCo, attempts bring similar closer. Specifically, first design two-level augmentations, including content-level structure-level, generate literally diverse pairs purposes. Then, fully exploit hierarchical information concepts, hierarchy-aware rank strategy (KHAR), ranks similarities between fine-grained manner. Next, adopt ranking learning task optimize our model based on augmented ranked questions. We conduct extensive experiments two real-world datasets. The experimental results demonstrate effectiveness model.